The invention relates to a classification method implemented in a layered neural network, comprising learning steps during which at least one layer is constructed by the addition of the successive neurons required for executing by successive dichotomies, a classification of examples divided into classes.
It also relates to a neural network implementing this method.
Neural networks are applied for classification problems, in particular for recognition of shapes and characters, and for speech signal processing, image processing, data compression, etc.
Neural networks are constituted from non-linear automatons generally connected to each other by synapses having synaptic coefficients. They allow the processing of problems which are difficult to process by conventional sequential computers.
The two commonest types of network are:
fully connected networks known as Hopfield networks
layered networks: the neurons are grouped in successive layers, each neuron is connected to all of the neurons of the following layer.
In the most general structure, information is fed forward from input terminals (passive) to an input layer (active) then successively to each hidden layer (active) until the output layer (active). In the most reduced structure, information is fed forward from input terminals (passive) to a single (=output) layer (active).
These systems are capable of learning by example or by self-organizing. The long computing times of sequential computers can be considerably reduced by the parallel carrying out of the operations which comprise learning phases and resolution phases.
In order to carry out a given operation, the neural networks must previously learn to carry it out. This phase, called the learning phase, makes use of examples. For many algorithms, the output results to be obtained with these examples are known in advance. Initially, the neural network which is not yet adapted to the intended task, will supply erroneous results. An error is then determined between the results obtained and those which should have been obtained and, on the basis of an adaptation criterion, the synaptic coefficients are modified in order to allow the neural network to learn the chosen example. This stage is reiterated over the batch of examples considered as necessary for a satisfactory learning of the neural network.
Learning algorithms are divided into two classes:
local learning, in which the modification of a synaptic coefficient C.sub.ij connecting a neuron j to a neuron i depends only on the information localized on the neurons i and j,
non-local learning, in which such modification depends on information located in the entire network. A well-known example is that of the backward propagation of the error in layered networks.
Various types of neural networks have been described in the article by R. P. LIPPMANN, "An introduction to computing with neural nets" IEEE ASSP Magazine, Apr. 1987 pp. 4 to 22.
In these neural networks the organization of the structure is fixed (in layers or fully connected) and the connections between neurons are fixed in advance.
The purpose of the learning which is then carried out is to seek an optimum structure by proposing different architectures and then making an a posteriori choice, as based on processing results.
An algorithm which allows the architecture to be determined during the learning has been proposed by: M. MEZARD and J. P. NADAL in "Learning in Feedforward Layered Networks: the tiling algorithm" J. Phys. A: Math. Gen. 22 (1989) pp. 2191-2203.
For this purpose the learning is initialized by optimizing the synaptic coefficients of a first neuron present in a particular layer and if this neuron does not suffice to accomplish the classification task an additional neuron is added in the current layer or in the following layer which is thus initialized. This approach allows the learning of the architecture and of the layered network parameters (Multilayer Perceptrons) while separating the outputs into two classes.
But such a neural network structure does not allow executing classification problems for examples distributed into several classes.